Method and apparatus for visualization of an environment of a motor vehicle

ABSTRACT

The invention relates to a method and an apparatus for visualizing an environment of a motor vehicle, wherein the apparatus has at least one camera for detecting an environment of the motor vehicle, at least one display unit, and an evaluation and control unit, which is designed to depict images of the environment on the display unit, depending on the pictures from the at least one camera, wherein the apparatus comprises a device for detecting a head position of a vehicle passenger, wherein the evaluation and control unit is designed so that the depiction on the display unit is adapted depending on the detected head position, so that the vehicle passenger sees a depiction of the environment of the motor vehicle that the vehicle passenger would see if the display unit were a transparent window of the motor vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of German Patent Application DE 102016 210 088.2, filed on Jun. 8, 2016 with the German Patent andTrademark Office. The contents of the aforesaid German PatentApplication is incorporated herein in its entirety for all purposes.

BACKGROUND

The invention relates to a method and an apparatus for visualization ofan environment of a motor vehicle.

Modern motor vehicles have a plurality of differently designed driverassistance systems. These can for example be designed as park assists,pedestrian protection systems, blind spot assists, or emergency brakeassists. Typically at least one detection apparatus is provided, whichdetects environmental information of the motor vehicle. The detectionapparatuses representing a motor vehicle environment sensor system arefor example designed as ultrasonic sensors, radar sensors, lidar sensorsor optical sensors. The at least one provided detection apparatus isoperationally connected to a control unit by means of which the detectedenvironmental information can be evaluated. In particular, correspondingsoftware can be provided, which enables image recognition so thatobjects in particular within the environment of the motor vehicle can bedetected and identified. This makes it possible to evaluate a drivingsituation of the motor vehicle, which in particular includes the earlydetection of an existing collision hazard.

With park assist, an area behind the motor vehicle is in particulardetected optically. Using the detected environmental information,adjustments are automatically adapted using the control unit, inparticular to change a yaw angle of the motor vehicle. To inform a motorvehicle driver, the environmental information describing the area behindthe motor vehicle is displayed on a central display apparatus, which inparticular can be integrated in an instrument panel of the motorvehicle.

A method is known from DE 10 2011 121 616 A1 in which at least onedetection direction of a detection apparatus used to determine anenvironmental information, or a direction describing a directioninformation toward an object described by the environmental informationstarting from a reference point, in particular a sitting position of adriver within the motor vehicle, is assigned to the environmentalinformation, wherein the at least one display apparatus covering anangular range in a horizontal plane is controlled to depict theenvironmental information within a subrange of the angular rangeassigned to the direction information.

The display apparatus is arranged within the vehicle and at leastpartially surrounds the edge of a passenger cell. The environmentalinformation is depicted on a display apparatus at a positioncorresponding to the subarea, so that the associated directioninformation can be detected very easily and intuitively by the driver.This is enabled by the extension of the display apparatus over theangular range in the horizontal plane. If for example a pendingcollision hazard is detected, which is approaching the motor vehiclefrom the direction with reference to the sitting position of the driver,the display apparatus is controlled in the subarea assigned to thedirection information in order to display the hazard. The detectiondirection of the detection apparatus designed as an optical sensor orcamera can be used in particular to determine the direction information.Alternatively, provision is made to detect the direction using theenvironmental information describing the environment of the motorvehicle, for example by means of image recognition. If for example theobject described by the environmental information is on a collisioncourse—defining the direction information—with the motor vehicle, thedisplay apparatus is controlled so that the environment information isdepicted in the subarea of the display apparatus assigned to thedirection information.

Preferably, the subarea of the display apparatus is controlled so thatthe output relative to the sitting position of the driver corresponds tothe direction toward the object described by the environmentalinformation. The sitting position of the driver defines the referencepoint for the output on the display apparatus. In particular, thedisplay apparatus must be controlled in a slightly different subarea ifthe sitting position of the driver changes. This can accordingly ensurethat the environmental information is output corresponding to thecorrect direction in which the object is located with reference to thesitting position. The sitting position is for example detected using asensor system which detect the seat settings, in particular withreference to a vehicle longitudinal and/or transverse axis. In addition,the tilt angle of a backrest is detected so that the sitting position ofthe driver can be determined as precisely as possible. Moreover, thesitting position of the driver can be detected using a camera arrangedwithin the vehicle that continuously detects the sitting position of thedriver during driving.

Preferably, the display apparatus is formed by a plurality of displayelements arranged adjacent to each other. The display elements are inparticular arranged on the side next to the vehicle window and/or in thearea of a vehicle door, vehicle column, in particular an A, B, or Ccolumn. The arrangement of at least one display element in the area ofthe vehicle column makes it possible in particular for environmentalinformation otherwise covered by the vehicle column to be displayed inthe associated subarea. Accordingly, the image detected by the imagingapparatus can be displayed in particular in the area of the vehiclecolumn, wherein the image corresponds to the detected area whichcorresponds to the area of the blind spot. The area of the blind spotthat cannot be seen due to the vehicle column is accordingly displayedin the area of the vehicle column, and the vehicle is accordinglyrendered figuratively transparent to the driver so that intersections inparticular can be easily seen.

A method is disclosed in DE 10 2008 034 606 A1 for depicting theenvironment of a vehicle on a mobile unit, which wirelessly receives atleast one image signal from the vehicle, wherein a depiction on themobile unit is generated by the at least one image signal which containsat least one virtual plane arranged in perspective on which is shown atleast one image, recorded with vehicle recording means, that comprisesat least part of the vehicle environment.

SUMMARY

The invention is based on an object of providing a method for depictingan environment of a motor vehicle that enables improved depiction of theenvironment. Another technical object is the provision of a suitableapparatus.

The above objects are solved by a method having the features of theindependent method claim and an apparatus having the features of thecorresponding independent apparatus claim. Additional embodiments willbecome apparent from the dependent claims and the following description.

DETAILED DESCRIPTION OF EMBODIMENTS

According to one exemplary aspect, a method for visualizing anenvironment of a motor vehicle is executed using at least one camera fordetecting an environment of the motor vehicle, at least one displayunit, and an evaluation and control unit. Images of the environment aredepicted on the display unit depending on the pictures provided by atleast one camera. The images of the environment do not necessarily haveto correspond to the pictures. Instead and in corresponding embodiments,an image can be composed of a previous picture and/or a plurality ofpictures from one or more cameras. In addition, the data of the picturesin an embodiment may be converted by suitable transformation matrices.Moreover, in an embodiment, at least one device may be provided fordetecting a head position of a vehicle passenger, wherein the depictionon the display unit is adapted depending on the detected head positionso that the vehicle passenger sees a depiction of the environment of themotor vehicle that the vehicle passenger would see if the display unitwere a transparent window of the motor vehicle.

The current aspect is based on the insight of the present inventors thatthe sitting position is insufficient as a reference point. In additionto the different height of the vehicle passengers, this is in particularrelevant when the head position changes. Intuitively, the vehiclepassenger anticipates a change in the depiction just as he would whenlooking out of the window. According to the current aspect, the fact isexploited that normally the field of vision is larger than the displayunit so that the image of the environment remains the same even when theviewing direction changes as long as the head position remains the same(even nodding or rotating the head does not change the image; only adifferent part of the image is in focus). Consequently, the vehiclepassenger receives an intuitive depiction of the vehicle environmentthat he would perceive if the motor vehicle were transparent like awindow in the area of the display unit.

In one embodiment, the head position is detected by at least oneinternal vehicle camera that for example is used to detect a state ofattention of a motor vehicle driver. However, the invention is notlimited to the motor vehicle driver. Accordingly, for example, theinvention can be used for a vehicle passenger in the back seat of thevehicle whose forward vision is blocked by a headrest. If a display unitis integrated in the headrest, it can also depict images which thepassenger would see if the display unit were a transparent windowthrough the headrest.

In principle and in these embodiments, all existing display units suchas a central display and/or an instrument cluster can be used as thedisplay unit.

In one embodiment, the display unit is a component of a mobile devicesuch as a smart phone. Since the alignment, or respectively position, ofthe display unit can change continuously with reference to the headposition, the position of the display unit is determined by an internalsensor system of the mobile device. Accordingly, one can look outsidethrough the motor vehicle by using the display unit, wherein theenvironment can be scanned by a movement of the display unit. Forexample, the display unit is aligned parallel to the vehicle floor sothat the vehicle environment below the vehicle is depicted. If themobile device is then moved back and forth, the ground below the vehiclecan be searched for objects, for example.

In another embodiment, pictures of the motor vehicle interior are takenwith a camera of the mobile device, wherein the position of the displayunit is determined by comparing the pictures with previously savedpictures of the interior. Since for example the precise position of acontrol element is known, the position toward the control element, andhence the position in space, can be determined from the picture of thecontrol element by the camera.

In addition or alternatively, the vehicle passenger head position can bedetected by the camera of the mobile device, wherein the position of thedisplay unit can be deduced using the head position determined by theinternal vehicle camera.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is explained in the following in more detail using afurther exemplary embodiment. The figures show:

FIG. 1 a schematic representation of a device for depicting/visualizingan environment of a motor vehicle,

FIG. 2 a schematic representation of a virtual field of vision,

FIG. 3 another schematic representation of a device fordepicting/visualizing an environment of a motor vehicle,

FIG. 4a a schematic representation with a mobile device,

FIG. 4b a schematic representation of the mobile device at a first pointin time, and

FIG. 4c a schematic representation of the mobile device at a secondpoint in time.

FIG. 1 schematically shows an apparatus 1 for depicting an environmentof a motor vehicle 100. The apparatus 1 comprises a plurality of cameras2 for detecting an environment of the motor vehicle 100. Moreover, theapparatus 1 comprises two display units 3 and 4, as well as two internalvehicle cameras 5 and 6. Finally, the apparatus 1 has an evaluation andcontrol unit 7. The display unit 4 is arranged in a headrest 8. Thedisplay unit 3 is for example an instrument cluster.

Using the interior vehicle camera 5, a head position 9 of a motorvehicle driver is determined and sent to the evaluation and control unit7. The evaluation and control unit 7 also discerns the position of thedisplay unit 3. Accordingly, the evaluation and control unit 7 discernsthe relative position, or respectively alignment between the head anddisplay unit 3. The evaluation and control unit 7 then determines avirtual field of vision that the motor vehicle driver would have if thedisplay unit were a transparent motor vehicle window. The evaluation andcontrol unit 7 then prepares the pictures of the camera 2 (includingpictures from past periods) such that an image of the vehicleenvironment is depicted on the display unit 3 that the motor vehicledriver would see if the display unit 3 were a transparent window. Forexample, the driver would see an object B located directly before themotor vehicle 100. The motor vehicle driver can then see this object Bon the display unit 3. The data transmission by the camera 2 to theevaluation and control unit 7 may, in corresponding embodiments, bewireless or wired.

Correspondingly, the interior vehicle camera 6 determines the headposition 10 of a vehicle passenger in the back seat of the motor vehicle100 and determines its relative position, or respectively alignment tothe display unit 4 (taking into account the adjusted height or headrest8). The vehicle environment then is depicted on the display unit 4 thatthe vehicle passenger would see if the display unit 4 were a transparentwindow. For example, an object A that would otherwise be covered by theheadrest 8 is depicted. The vehicle passenger therefore looks clearlythrough the headrest 8. In one example, the vehicle passenger can selectthe depiction of the vehicle environment by a selection menu andotherwise use the display unit 4 for other purposes. It may also beprovided for the vehicle passenger to be able to change the depiction bybeing able to manually adjust the virtual field of vision.

FIG. 2 schematically shows such a virtual field of vision vS for thedisplay unit 3. The real field of vision rS of a camera 2 (for exampleof the middle, front camera 2 according to FIG. 1) is also shown. As canbe seen in the FIG., the overlap between the virtual field of vision vSand the real field of vision rS is slight. However, the virtual field ofvision vS can also be determined from previous pictures by camera 2.

FIG. 3 depicts the apparatus 1 in an alternative embodiment, wherein itcan also be entirely or partially combined with the apparatus 1according to FIG. 1. The same elements refer to the same referencenumerals. In contrast to FIG. 1, a display unit 11 is provided as acomponent of a mobile device 12. The mobile device 12 has a camera 13and communicates wirelessly with the evaluation and control unit 7.Using the camera 13, the mobile device 12 detects either the headposition 9 of the motor vehicle driver or an interior trim 14 of themotor vehicle 100. The position of the mobile device 12 and hence therelative location to the head position 9 is determined using thecomparison with the determined head position 9 by means of the interiorvehicle camera 5, or by comparing with previously saved position data ofthe interior trim 14. Accordingly, the display unit 11 is used as avirtual window for depicting the vehicle environment.

This will be briefly explained with reference to FIGS. 4a to 4 c. Themotor vehicle driver holds his mobile device 12 parallel to a vehiclefloor 15. An object C is below the vehicle floor 15 on a street 16. Themobile device 12 then receives data from the evaluation and control unit7 depending on the position of the mobile device 12 and the headposition 9. The vehicle passenger then sees the object C on the displayunit 11 as if he were looking through the motor vehicle with the displayunit 11 (see FIG. 4b ). If the motor vehicle driver then moves themobile device to the left toward the interior vehicle wall, thedepiction of the object C correspondingly moves to the right on thedisplay unit 11 (see FIG. 4c ). The vehicle passenger can scan theenvironment by moving the mobile device 12 and/or changing his headposition. It should be noted that parts of the motor vehicle 100 (suchas the wheels 17) can also be depicted on the display unit 11 for betterorientation.

Other variations to the disclosed embodiments can be understood andeffected by those skilled in the art in practicing the claimedinvention, from a study of the drawings, the disclosure, and theappended claims. In the claims, the word “comprising” does not excludeother elements or steps, and the indefinite article “a” or “an” does notexclude a plurality. A single processor, module or other unit mayfulfill the functions of several items recited in the claims.

The mere fact that certain measures are recited in mutually differentdependent claims or embodiments does not indicate that a combination ofthese measured cannot be used to advantage. A computer program may bestored/distributed on a suitable medium, such as an optical storagemedium or a solid-state medium supplied together with or as part ofother hardware, but may also be distributed in other forms, such as viathe Internet or other wired or wireless telecommunication systems. Anyreference signs in the claims should not be construed as limiting thescope.

What is claimed is:
 1. A method for visualizing an environment of amotor vehicle using at least one camera for detecting an environment ofthe motor vehicle, at least one display unit, and an evaluation andcontrol unit, which depicts images of the environment on the displayunit in dependence of pictures from the at least one camera, wherein atleast one device is provided for detecting a head position of a vehiclepassenger, wherein the depiction on the display unit is adapteddepending on the detected head position and the position of the displayunit, so that the vehicle passenger sees a depiction of the environmentof the motor vehicle that the vehicle passenger would see if the displayunit were a transparent window of the motor vehicle.
 2. The methodaccording to claim 1, wherein the head position is detected by at leastone vehicle passenger camera.
 3. The method according to claim 1,wherein the display unit is a component of a mobile device, wherein theposition of the display unit is determined by an internal sensor system.4. The method according to claim 3, wherein the mobile device takespictures of the interior with a camera of the mobile device, and theposition of the display unit is determined by comparing the pictureswith previously saved pictures of the interior.
 5. The method accordingto claim 3, wherein the mobile device detects the head position of thevehicle passenger with a camera of the mobile device, wherein theposition of the display unit is deduced using the determined headposition by a vehicle passenger camera.
 6. An apparatus for visualizingan environment of a motor vehicle, wherein the apparatus comprises atleast one camera for detecting an environment of the motor vehicle, atleast one display unit, and an evaluation and control unit, which isdesigned to depict images of the environment on the display unitdepending on the pictures from the at least one camera, wherein theapparatus further comprises a device for detecting a head position of avehicle passenger, wherein the evaluation and control unit is configuredso that the depiction on the display unit is adapted depending on thedetected head position, so that the vehicle passenger sees a depictionof the environment of the motor vehicle that the vehicle passenger wouldsee if the display unit were a transparent window of the motor vehicle.7. The apparatus according to claim 6, comprising at least one interiorvehicle camera, which is configured to determine the head position of avehicle passenger.
 8. The apparatus according to claim 6, wherein the atleast one display unit is a component of a mobile device, and/or atleast one display unit is arranged in the motor vehicle.
 9. Theapparatus according to claim 6, wherein the mobile device is configuredwith an internal sensor system for detecting the position of the displayunit.
 10. The apparatus according to claim 8, wherein the mobile devicecomprises at least one camera, wherein the mobile device or theevaluation and control unit is configured so that the position of thedisplay unit is determined from pictures of the camera of the interiorof the motor vehicle and by comparing with previously saved pictures ofthe interior, or that the camera detects the head position of thevehicle passenger, wherein the position of the display unit is deducedfrom the determined head position by a vehicle passenger camera.
 11. Themethod according to claim 2, wherein the display unit is a component ofa mobile device, wherein the position of the display unit is determinedby an internal sensor system.
 12. The method according to claim 4,wherein the mobile device detects the head position of the vehiclepassenger with a camera of the mobile device, wherein the position ofthe display unit is deduced using the determined head position by avehicle passenger camera.
 13. The apparatus according to claim 7,wherein the at least one display unit is a component of a mobile device,and/or at least one display unit is arranged in the motor vehicle. 14.The apparatus according to claim 7, wherein the mobile device isconfigured with an internal sensor system for detecting the position ofthe display unit.
 15. The apparatus according to claim 8, wherein themobile device is configured with an internal sensor system for detectingthe position of the display unit.
 16. The apparatus according to claim9, wherein the mobile device comprises at least one camera, wherein themobile device or the evaluation and control unit is configured so thatthe position of the display unit is determined from pictures of thecamera of the interior of the motor vehicle and by comparing withpreviously saved pictures of the interior, or that the camera detectsthe head position of the vehicle passenger, wherein the position of thedisplay unit is deduced from the determined head position by a vehiclepassenger camera.